Do masks cover more than just a face? A study on how facemasks affect the perception of emotional expressions according to their degree of intensity.

Emotional facial expressions convey crucial information in nonverbal communication and serve as a mediator in face-to-face relationships. Their recognition would rely on specific facial traits depending on the perceived emotion. During the COVID-19 pandemic, wearing a facemask has thus disrupted the human ability to read emotions from faces. Yet, these effects are usually assessed across studies from faces expressing stereotypical and exaggerated emotions, which is far removed from real-life conditions. The objective of the present study was to evaluate the impact of facemasks through an emotion categorization task using morphs ranging from a neutral face and an expressive face (anger, disgust, fear, happiness, and sadness) (from 0% neutral to 100% expressive in 20% steps). Our results revealed a strong impact of facemasks on the recognition of expressions of disgust, happiness, and sadness, resulting in a decrease in performance and an increase in misinterpretations, both for low and high levels of intensity. In contrast, the recognition of anger and fear, as well as neutral expression, was found to be less impacted by mask-wearing. Future studies should address this issue from a more ecological point of view with the aim of taking concrete adaptive measures in the context of daily interactions.

The time course of categorical perception of facial expressions.

Decoding emotions on others' faces is one of the most important functions of the human brain, which has been widely studied in cognitive neuroscience. However, the precise time course of facial expression categorization in the human brain is still a matter of debate. Here we used an original paradigm to measure categorical perception of facial expression changes during event-related potentials (ERPs) recording, in which a face stimulus dynamically switched either to a different expression (between-category condition) or to the same expression (within-category condition), the physical distance between the two successive faces being equal across conditions. The switch between faces generated a negative differential potential peaking at around 160 ms over occipito-temporal regions, similar in term of latency and topography to the well-known face-selective N170 component. This response was larger in the condition where the switch occurred between faces that were perceived as having different facial expressions compared to the same expression. In addition, happy expressions were categorized around 20 ms faster than fearful expressions (respectively, 135 and 156 ms). These findings provide evidence that changes of facial expressions are categorically perceived as early as 160 ms following stimulus onset over the occipito-temporal cortex.

The rapid and automatic categorization of facial expression changes in highly variable natural images.

Emotional expressions are quickly and automatically read from human faces under natural viewing conditions. Yet, categorization of facial expressions is typically measured in experimental contexts with homogenous sets of face stimuli. Here we evaluated how the 6 basic facial emotions (Fear, Disgust, Happiness, Anger, Surprise or Sadness) can be rapidly and automatically categorized with faces varying in head orientation, lighting condition, identity, gender, age, ethnic origin and background context. High-density electroencephalography was recorded in 17 participants viewing 50 s sequences with natural variable images of neutral-expression faces alternating at a 6 Hz rate. Every five stimuli (1.2 Hz), variable natural images of one of the six basic expressions were presented. Despite the wide physical variability across images, a significant F/5 = 1.2 Hz response and its harmonics (e.g., 2F/5 = 2.4 Hz, etc.) was observed for all expression changes at the group-level and in every individual participant. Facial categorization responses were found mainly over occipito-temporal sites, with distinct hemispheric lateralization and cortical topographies according to the different expressions. Specifically, a stronger response was found to Sadness categorization, especially over the left hemisphere, as compared to Fear and Happiness, together with a right hemispheric dominance for categorization of Fearful faces. Importantly, these differences were specific to upright faces, ruling out the contribution of low-level visual cues. Overall, these observations point to robust rapid and automatic facial expression categorization processes in the human brain.

The N170 is Sensitive to Long-term (Personal) Familiarity of a Face Identity.

The N170 is a large deflection of the human electroencephalogram (EEG), peaking at about 170 milliseconds over the occipito-temporal cortex after the sudden onset of a face stimulus. The N170 reflects perceptual awareness of a face and its onset corresponds to the emergence of reliable face-selectivity in the human brain. However, whether sensitivity to the long-term familiarity of a face identity emerges already at this early time-point remains debated. Here we provide a brief survey of the 45 published studies comparing the N170 response to unfamiliar and familiar (famous, experimentally familiarized, personally familiar and own) faces. Even though effects of familiarity on the N170 are relatively small and inconsistent across studies, this overview indicates that face familiarity significantly increases the N170 amplitude. This effect is especially present for personally familiar faces, learned in natural conditions. In the human brain, effects linked to familiarity with specific facial identities therefore appear to emerge between 150 and 200 ms in occipito-temporal brain regions, i.e., shortly after the onset of face-selectivity but at the same time as the earliest high-level effects of immediate unfamiliar face identity repetition. This observation challenges standard neurocognitive models with a clear-cut distinction between perceptual and memory stages in human face recognition.

Time course of spatial frequency integration in face perception: An ERP study.

Face perception is based on the processing and integration of multiple spatial frequency (SF) ranges. However, the temporal dynamics of SF integration to form an early face representation in the human brain is still a matter of debate. To address this issue, we recorded event-related potentials (ERPs) during the presentation of spatial frequency-manipulated facial images. Twenty-six participants performed a gender discrimination task on non-filtered, low-, high-, and band-pass filtered face images, corresponding, respectively, to the full range, spatial frequencies up to 8 cycles/image, above 32 cycles/image, and from 8 to 16 cycles/image. Behaviorally, the task related-performance was more accurate and faster for non-filtered (NF) and mid-range SF (MSF) than for low SF (LSF) and high SF (HSF) stimuli. At both behavioral and electrophysiological levels, response to MSF contained in faces did not differ from the responses to full spectrum non-filtered (NF) facial images. In ERPs, LSF facial images evoked the largest P1 amplitude while HSF facial images evoked the largest N170 amplitude compared with the other three conditions. Since LSFs and HSFs would transmit global and local information respectively, our observations lend further support to the "coarse-to-fine" processing theory of faces. Furthermore, they offer original evidence of the effectiveness and adequacy of the mid-range spatial frequency in face perception. Possible theoretical interpretations of our findings are discussed.

Factors influencing spatial frequency extraction in faces: A review.

JEANTET, C., Caharel, S., Schwan, R., Lighezzolo-Alnot, J., and Laprevote, V. Factors influencing spatial frequencies extraction in faces: a review. NEUROSCI BIOBEHAV REV XX(X) XXX-XXX, 2017. Spatial frequency is an elementary aspect of visual perception. Moreover, faces elicit distinct responses by the human visual system when compared to other visual objects. This review examines the factors influencing spatial frequency processing in faces. Visual perception of a face results from the interaction between the physical properties of the stimulus and the human visual system. We first review the methodology of visual stimulus production and presentation in the assessment of spatial frequency processing in faces. Image properties interact with the physical constraints of the visual system. Other cognitive phenomena also drive the processing of spatial frequencies in faces. Finally, the observer's characteristics may further influence this spatial processing. Overall, the studies indicate that spatial frequency processing in faces is not a fixed process, conditioned by physical constraints alone, but a flexible process, dependent of various cognitive constraints, developmental, and health conditions. Finally, limitations and new challenges are discussed.

Short-term EEG dynamics and neural generators evoked by navigational images.

The ecological environment offered by virtual reality is primarily supported by visual information. The different image contents and their rhythmic presentation imply specific bottom-up and top-down processing. Because these processes already occur during passive observation we studied the brain responses evoked by the presentation of specific 3D virtual tunnels with respect to 2D checkerboard. For this, we characterized electroencephalograhy dynamics (EEG), the evoked potentials and related neural generators involved in various visual paradigms. Time-frequency analysis showed modulation of alpha-beta oscillations indicating the presence of stronger prediction and after-effects of the 3D-tunnel with respect to the checkerboard. Whatever the presented image, the generators of the P100 were situated bilaterally in the occipital cortex (BA18, BA19) and in the right inferior temporal cortex (BA20). In checkerboard but not 3D-tunnel presentation, the left fusiform gyrus (BA37) was additionally recruited. P200 generators were situated in the temporal cortex (BA21) and the cerebellum (lobule VI/Crus I) specifically for the checkerboard while the right parahippocampal gyrus (BA36) and the cerebellum (lobule IV/V and IX/X) were involved only during the 3D-tunnel presentation. For both type of image, P300 generators were localized in BA37 but also in BA19, the right BA21 and the cerebellar lobule VI for only the checkerboard and the left BA20-BA21 for only the 3D-tunnel. Stronger P300 delta-theta oscillations recorded in this later situation point to a prevalence of the effect of changing direction over the proper visual content of the 3D-tunnel. The parahippocampal gyrus (BA36) implicated in navigation was also identified when the 3D-tunnel was compared to their scrambled versions, highlighting an action-oriented effect linked to navigational content.

Emotional contexts modulate intentional memory suppression of neutral faces: Insights from ERPs.

The main goal of present work is to gain new insight into the temporal dynamics underlying the voluntary memory control for neutral faces associated with neutral, positive and negative contexts. A directed forgetting (DF) procedure was used during the recording of EEG to answer the question whether is it possible to forget a face that has been encoded within a particular emotional context. A face-scene phase in which a neutral face was showed in a neutral or emotional scene (positive, negative) was followed by the voluntary memory cue (cue phase) indicating whether the face had to-be remember or to-be-forgotten (TBR and TBF). Memory for faces was then assessed with an old/new recognition task. Behaviorally, we found that it is harder to suppress faces-in-positive-scenes compared to faces-in-negative and neutral-scenes. The temporal information obtained by the ERPs showed: 1) during the face-scene phase, the Late Positive Potential (LPP), which indexes motivated emotional attention, was larger for faces-in-negative-scenes compared to faces-in-neutral-scenes. 2) Remarkably, during the cue phase, ERPs were significantly modulated by the emotional contexts. Faces-in-neutral scenes showed an ERP pattern that has been typically associated to DF effect whereas faces-in-positive-scenes elicited the reverse ERP pattern. Faces-in-negative scenes did not show differences in the DF-related neural activities but larger N1 amplitude for TBF vs. TBR faces may index early attentional deployment. These results support the hypothesis that the pleasantness or unpleasantness of the contexts (through attentional broadening and narrowing mechanisms, respectively) may modulate the effectiveness of intentional memory suppression for neutral information.

The early visual encoding of a face (N170) is viewpoint-dependent: a parametric ERP-adaptation study.

Visual representations of faces are extracted shortly after 100 ms in the human brain, leading to an occipito-temporal cortex N170 event-related potential (ERP). To understand the nature of this early visual representation, a full-front adapting face preceded a different or identical target face identity. The target face varied parametrically in head orientation from the adapting face (0-90°, 15° steps). The N170 elicited by the target face increased progressively from 0° up to 30° head orientation, with no further increase until 90°. The N170 decreased for repeated face identities, this effect being stable between 0° and 30° changes of viewpoint, and no effect beyond that angle. These observations suggest that a face is encoded in a view-dependent manner, being matched to either a full-front or a profile face view. Yet, individual face representations activated as early as the peak of the N170 generalize partially across views.

Face familiarity decisions take 200 msec in the human brain: electrophysiological evidence from a go/no-go speeded task.

Recognizing a familiar face rapidly is a fundamental human brain function. Here we used scalp EEG to determine the minimal time needed to classify a face as personally familiar or unfamiliar. Go (familiar) and no-go (unfamiliar) responses elicited clear differential waveforms from 210 msec onward, this difference being first observed at right occipito-temporal electrode sites. Similar but delayed (by about 40 msec) responses were observed when go response were required to the unfamiliar rather than familiar faces, in a second group of participants. In both groups, a small increase of amplitude was also observed on the right hemisphere N170 face-sensitive component for familiar faces. However, unlike the post-200 msec differential go/no-go effect, this effect was unrelated to behavior and disappeared with repetition of unfamiliar faces. These observations indicate that accumulation of evidence within the first 200 msec poststimulus onset is sufficient for the human brain to decide whether a person is familiar based on his or her face, a time frame that puts strong constraints on the time course of face processing.

Early holistic face-like processing of Arcimboldo paintings in the right occipito-temporal cortex: evidence from the N170 ERP component.

The properties of the face-sensitive N170 component of the event-related brain potential (ERP) were explored through an orientation discrimination task using natural faces, objects, and Arcimboldo paintings presented upright or inverted. Because Arcimboldo paintings are composed of non-face objects but have a global face configuration, they provide great control to disentangle high-level face-like or object-like visual processes at the level of the N170, and may help to examine the implication of each hemisphere in the global/holistic processing of face formats. For upright position, N170 amplitudes in the right occipito-temporal region did not differ between natural faces and Arcimboldo paintings but were larger for both of these categories than for objects, supporting the view that as early as the N170 time-window, the right hemisphere is involved in holistic perceptual processing of face-like configurations irrespective of their features. Conversely, in the left hemisphere, N170 amplitudes differed between Arcimboldo portraits and natural faces, suggesting that this hemisphere processes local facial features. For upside-down orientation in both hemispheres, N170 amplitudes did not differ between Arcimboldo paintings and objects, but were reduced for both categories compared to natural faces, indicating that the disruption of holistic processing with inversion leads to an object-like processing of Arcimboldo paintings due to the lack of local facial features. Overall, these results provide evidence that global/holistic perceptual processing of faces and face-like formats involves the right hemisphere as early as the N170 time-window, and that the local processing of face features is rather implemented in the left hemisphere.

Asymmetric switch-costs and ERPs reveal facial identity dominance over expression.

Previous studies on face processing have revealed an asymmetric overlap between identity and expression, as identity is processed irrespective of expression while expression processing partly depends on identity. To investigate whether this relative interaction is caused by dominance of identity over expression, participants performed familiarity and expression judgments during task switching. This paradigm reveals task-set dominance with a paradoxical asymmetric switch-cost (i.e., greater difference between switch and repeat trials when switching toward the dominant task). Event-related potentials (ERPs) were recorded to find the neural signature of the asymmetric cost. As expected, greater switch-cost was shown in the familiarity task with respect to response times, indicating its dominance over the expression task. Moreover, a left-sided ERP correlate of this effect was observed at the level of the frontal N2 component, interpreted as an index of modulations in endogenous executive control. Altogether, these results confirm the overlap between identity and expression during face processing and further indicate their relative dominance.

The speed of recognition of personally familiar faces.

Despite the generally accepted notion that humans are very good and fast at recognising familiar individuals from their faces, the actual speed with which this fundamental brain function can be achieved remains largely unknown. Here, two groups of participants were required to respond by finger-lift when presented with either a photograph of a personally familiar face (classmate), or an unfamiliar one. This speeded manual go/no-go categorisation task revealed that personally familiar faces could be categorised as early as 380 ms after presentation, about 80 ms faster than unfamiliar faces. When response times were averaged across all 8 stimulus presentations, we found that minimum RTs for both familiar and unfamiliar face decisions were substantially lower (310 ms and 370 ms). Analyses confirmed that stimulus repetition enhanced the speed with which faces were categorised, irrespective of familiarity, and that repetition did not affect the observed benefit in RTS for familiar over unfamiliar faces. These data, representing the elapsed time from stimulus onset to motor output, put constraints on the time course of familiar face recognition in the human brain, which can be tracked more precisely by high temporal resolution electrophysiological measures.

ERP evidence for the speed of face categorization in the human brain: Disentangling the contribution of low-level visual cues from face perception.

How fast are visual stimuli categorized as faces by the human brain? Because of their high temporal resolution and the possibility to record simultaneously from the whole brain, electromagnetic scalp measurements should be the ideal method to clarify this issue. However, this question remains debated, with studies reporting face-sensitive responses varying from 50 ms to 200 ms following stimulus onset. Here we disentangle the contribution of the information associated with the phenomenological experience of a face (phase) from low-level visual cues (amplitude spectrum, color) in accounting for early face-sensitivity in the human brain. Pictures of faces and of a category of familiar objects (cars), as well as their phase-scrambled versions, were presented to fifteen human participants tested with high-density (128 channels) EEG. We replicated an early face-sensitivity - larger response to pictures of faces than cars - at the level of the occipital event-related potential (ERP) P1 (80- ). However, a similar larger P1 to phase-scrambled faces than phase-scrambled cars was also found. In contrast, the occipito-temporal N170 was much larger in amplitude for pictures of intact faces than cars, especially in the right hemisphere, while the small N170 elicited by phase-scrambled stimuli did not differ for faces and cars. These findings show that sensitivity to faces on the visual evoked potentials P1 and N1 (N170) is functionally dissociated: the P1 face-sensitivity is driven by low-level visual cues while the N1 (or N170) face-sensitivity reflects the perception of a face. Altogether, these observations indicate that the earliest access to a high-level face representation, that is, a face percept, does not precede the N170 onset in the human brain. Furthermore, they allow resolving apparent discrepancies between the timing of rapid human saccades towards faces and the early activation of high-level facial representations as shown by electrophysiological studies in the primate brain. More generally, they put strong constraints on the interpretation of early (before 100 ms) face-sensitive effects in the human brain.

Early electrophysiological correlates of adaptation to personally familiar and unfamiliar faces across viewpoint changes.

Behavioral studies have shown that matching individual faces across depth rotation is easier and faster for familiar than unfamiliar faces. Here we used event-related potentials (ERPs) to clarify the locus of this behavioral facilitation, that is whether it reflects changes at the level of perceptual face encoding, or rather at later stages of processing. We used an identity adaptation paradigm in ERPs, during which a first (adapting) face (~3000 ms) rotated 30° in depth was followed by a second full front face (200 ms) which was either the same or a different identity as the first face. For unfamiliar faces, the early face-sensitive N170 component was reduced for immediately repeated as compared to different unfamiliar faces in the right hemisphere only. However, for personally familiar faces, the effect was absent at right hemisphere electrode sites and appeared instead over the left hemisphere at the same latency. Later effects of face identity adaptation were also present on the scalp, but from about 300 to 400 ms over fronto-central regions, and slightly later on occipito-temporal regions, there was a strong adaptation effect only for familiar faces. These observations suggest that the perceptual encoding of familiar and unfamiliar faces may be of different nature, as indicated by early (N170) hemispheric differences for identity adaptation effects depending on long-term familiarity. However, the behavioral advantage provided by familiarity to match faces across viewpoints might rather be related to processes that are closer in time to the behavioral response, such as semantic associations between the faces to match.

Other-race and inversion effects during the structural encoding stage of face processing in a race categorization task: an event-related brain potential study.

To investigate the mechanisms underlying the other-race effect, in particular at what stage of face processing differences between same-race (SR) and other-race (OR) stimuli occur, electrophysiological and behavioral data were obtained on Caucasian participants viewing photographs of Caucasian, Asian, and African faces in upright and inverted orientations. During a race categorization task, reaction times were faster for African than Asian faces, and both of them faster than Caucasian ones, independent of their orientation. The face-sensitive N170 component was low in amplitude for Caucasian, intermediate for Asian, and maximal for African faces. The face inversion effect was observed for all ethnic groups on N170 amplitudes, but was more evident for Caucasian faces. According to the perceptual expertise hypothesis, our results indicate that SR faces involve more configural/holistic processing OR faces.

Early (n170/m170) face-sensitivity despite right lateral occipital brain damage in acquired prosopagnosia.

Compared to objects, pictures of faces elicit a larger early electromagnetic response at occipito-temporal sites on the human scalp, with an onset of 130 ms and a peak at about 170 ms. This N170 face effect is larger in the right than the left hemisphere and has been associated with the early categorization of the stimulus as a face. Here we tested whether this effect can be observed in the absence of some of the visual areas showing a preferential response to faces as typically identified in neuroimaging. Event-related potentials were recorded in response to faces, cars, and their phase-scrambled versions in a well-known brain-damaged case of prosopagnosia (PS). Despite the patient's right inferior occipital gyrus lesion encompassing the most posterior cortical area showing preferential response to faces ("occipital face area"), we identified an early face-sensitive component over the right occipito-temporal hemisphere of the patient that was identified as the N170. A second experiment supported this conclusion, showing the typical N170 increase of latency and amplitude in response to inverted faces. In contrast, there was no N170 in the left hemisphere, where PS has a lesion to the middle fusiform gyrus and shows no evidence of face-preferential response in neuroimaging (no left "fusiform face area"). These results were replicated by a magnetoencephalographic investigation of the patient, disclosing a M170 component only in the right hemisphere. These observations indicate that face-preferential activation in the inferior occipital cortex is not necessary to elicit early visual responses associated with face perception (N170/M170) on the human scalp. These results further suggest that when the right inferior occipital cortex is damaged, the integrity of the middle fusiform gyrus and/or the superior temporal sulcus - two areas showing face-preferential responses in the patient's right hemisphere - might be necessary to generate the N170 effect.

Perceptual interactions between visual processing of facial familiarity and emotional expression: an event-related potentials study during task-switching.

Models of face processing suggest that facial familiarity and expression processes involve independent visual systems. But under some conditions, the two processes interact, as when selective attention is solicited, and/or when a link is established between consecutive stimuli. To assess these assumptions during perceptual face processing, event-related potentials (ERPs) were used while subjects discriminated either familiarity or expression in a task-switching paradigm. Switched trials were designed with competitor priming, the unattended dimension being previously attended. The results indicate interactions appearing in the right hemisphere during the perceptual encoding stage (N170) when subjects processed either familiarity or expression during switched trials. These interactions gain both hemispheres during memory retrieval (P2) and in terms of accuracy. Altogether, these results confirm the critical role of the right hemisphere in perceiving faces and their expressions. Moreover, they suggest that familiarity and expression can interact in both directions.

Recognizing an individual face: 3D shape contributes earlier than 2D surface reflectance information.

The human brain recognizes faces by means of two main diagnostic sources of information: three-dimensional (3D) shape and two-dimensional (2D) surface reflectance. Here we used event-related potentials (ERPs) in a face adaptation paradigm to examine the time-course of processing for these two types of information. With a 3D morphable model, we generated pairs of faces that were either identical, varied in 3D shape only, in 2D surface reflectance only, or in both. Sixteen human observers discriminated individual faces in these 4 types of pairs, in which a first (adapting) face was followed shortly by a second (test) face. Behaviorally, observers were as accurate and as fast for discriminating individual faces based on either 3D shape or 2D surface reflectance alone, but were faster when both sources of information were present. As early as the face-sensitive N170 component (approximately 160 ms following the test face), there was larger amplitude for changes in 3D shape relative to the repetition of the same face, especially over the right occipito-temporal electrodes. However, changes in 2D reflectance between the adapter and target face did not increase the N170 amplitude. At about 250 ms, both 3D shape and 2D reflectance contributed equally, and the largest difference in amplitude compared to the repetition of the same face was found when both 3D shape and 2D reflectance were combined, in line with observers' behavior. These observations indicate that evidence to recognize individual faces accumulate faster in the right hemisphere human visual cortex from diagnostic 3D shape information than from 2D surface reflectance information.

Early adaptation to repeated unfamiliar faces across viewpoint changes in the right hemisphere: evidence from the N170 ERP component.

Event-related potential (ERP) studies have shown that sensitivity to individual faces emerges as early as approximately 160ms in the human occipitotemporal cortex (N170). Here we tested whether this effect generalizes across changes in viewpoint. We recorded ERPs during an unfamiliar individual face adaptation paradigm. Participants were presented first with an adapting face ( approximately 3000ms) rotated 30 degrees in depth, followed by a second face (200ms) in a frontal view of either the same or a different identity. The N170 amplitude at right occipitotemporal sites to the second stimulus was reduced for repeated as compared to different faces. A bilateral adaptation effect emerged after 250ms following stimulus onset. These observations indicate that individual face representations activated as early as 160ms after stimulus onset in the right hemisphere show a substantial degree of generalization across viewpoints.